Maximum Likelihood Estimation for Lossy Data Compression∗

In lossless data compression, given a sequence of observations (Xn)n≥1 and a family of probability distributions {Qθ}θ∈Θ, the estimators (θ̃n)n≥1 obtained by minimizing the ideal Shannon code-lengths over the family {Qθ}θ∈Θ, θ̃n := arg min θ∈Θ [ − logQθ(X 1 ) ] , whereXn 1 := (X1, X2, . . . , Xn), coincide with the classical maximum-likelihood estimators (MLEs). In the corresponding lossy compression setting, the ideal Shannon code-lengths are approximately − logQθ(B(X 1 , D)) bits, where B(Xn 1 , D) is the distortion-ball of radius D around the source sequence Xn 1 . In this work we consider the analogous estimators obtained by minimizing these lossy code-lengths, θ̂n := arg min θ∈Θ [ − logQθ(B(X 1 , D)) ] . The θ̂n are a lossy version of the MLEs, which we call “lossy MLEs”. We investigate the strong consistency of lossy MLEs when the Qθ are i.i.d. and the sequence (Xn)n≥1 is stationary and ergodic.